Traffic engineering system and method

ABSTRACT

The present invention is directed to a novel system and method for traffic engineering in a packet-switched network, such as an Internet Protocol (“IP”) based backbone network. A global view of the network is constructed utilizing a network data model that can be readily constructed from the balkanized network information associated locally with the individual elements in the network. The data model, in turn, can be utilized to support useful traffic engineering tools such as routing modeling and visualization.

FIELD OF THE INVENTION

[0001] The present invention relates generally to packet-switchednetworks. More particularly, the present invention relates to trafficengineering in a packet-switched network.

BACKGROUND OF THE INVENTION

[0002] The Internet is divided into a collection of autonomous systems,each autonomous system (“AS”) managed by an Internet Service Provider(“ISP”) who operates a backbone network that connects to customers andother service providers. Large ISPs have few software systems and toolsto support traffic measurement and network modeling, the underpinningsof effective traffic engineering. Seemingly simple questions abouttopology, traffic, and routing are surprisingly hard to answer intoday's packet-switched networks. A tremendous amount of work has goneinto developing mechanisms and protocols for controlling traffic. Bycomparison, little work has gone to support traffic measurement andnetwork modeling in operational networks. Unfortunately, unless controlmechanisms are driven by the appropriate measurements and understandingfrom well-tested models, the benefit of the controls will be limited.

[0003] Accordingly, there is a need for new systems and methods ofmeasuring and modeling a packet-switched network that permit effectivetraffic engineering.

SUMMARY OF THE INVENTION

[0004] It is an object of the present invention to enable networkmanagement tools in a packet-switched network that are capable ofefficient reporting, capacity planning, provisioning, configurationdebugging, performance debugging, and allowing the investigation of theimpact of evolutionary changes to the network.

[0005] Thus, the present invention is directed to a novel system andmethod for traffic engineering in a packet-switched network, such as anInternet Protocol (“IP”) based backbone network. A global view of thenetwork is constructed utilizing a network data model that can bereadily constructed from the balkanized network information associatedlocally with the individual elements in the network. The data model, inturn, can be utilized to support useful traffic engineering tools suchas routing modeling and visualization. Unlike conventionalcircuit-switching, Frame Relay or ATM networks, in which global views oftopology and traffic are either given or trivial to derive, basicconcepts of traffic engineering such as a traffic matrix or anoffered-load are simply not present in IP networks and must be estimatedand/or derived. Moreover, they must be derived in a manner that takesinto account the dynamic set of intra-domain and inter-domain routingprotocols.

[0006] These and other advantages of the invention will be apparent tothose of ordinary skill in the art by reference to the followingdetailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007]FIG. 1 sets forth an abstract software architecture, in accordancewith a preferred embodiment of the present invention.

[0008]FIG. 2 sets forth a diagram of an IP backbone network.

[0009]FIG. 3 and FIG. 4 set forth pseudo-code for computing trafficflows, in accordance with a preferred embodiment of the presentinvention.

[0010]FIG. 5 sets forth pseudo-code for a disambiguation process, inaccordance with a preferred embodiment of the present invention.

[0011]FIG. 6 set forth a diagram illustrating routing traffic division,in accordance with a preferred embodiment of the present invention.

[0012]FIG. 7 through FIG. 11 set forth examples of graphical userinterfaces for visualization of the network, in accordance with apreferred embodiment of the present invention.

DETAILED DESCRIPTION

[0013]FIG. 1 sets forth an abstract architecture of an advanced trafficengineering software tool configured in accordance with a preferredembodiment of the present invention. The software is advantageouslydivided into separate modules, each of which can be implemented in anyof a number of suitable programming languages capable of execution on adigital computer, such as C, Tcl/TM, or Perl. An advantageous manner ofprogram code development is to use a software package, such as “Obj2C”,to convert data object descriptions into generated programming code forstandard manipulation of the data objects and a prototype graphical userinterface.

[0014] As is further described in the following sections below, thesoftware entitled by the inventors “NetScope” comprises three modules: adata model module 110 responsible for construction and manipulation ofthe network data model, a routing model module 120 responsible forconstruction and manipulation of the network routing model, and avisualization module 130 responsible for visualization, display, andcorrelation of multiple views of the network and usage information. Thedata model module 110 receives, as input, network topology informationand network traffic demand information in order to properly populate thenetwork data model. Netscope is advantageously separated and modularized(as shown abstractly by the dotted line) from the sources of the networktopology and traffic demand information. This architecture permits theparts above the dotted line to be unaware of changes to modules belowthe dotted line. The decomposition into modules and the design of theunderlying modules localize possible changes to the network, allowingfor the simple evolution and extension of the software.

[0015] One way of obtaining the configuration and traffic data is shownin FIG. 1: namely, two other modules 140 and 150 entitled by theinventors “NetBuild” and “DataBuild” respectively. The modules 140 and150 provide input to the data model module 110; these two modules takeas input the raw data regarding network configuration and measurement,and provide as output higher level abstractions and information for thedata model. This is highly advantageous in that an operational networkis under continuous change. It should be noted that the inputs to thedata model module 110 need not come from an operational network. Forexample, configuration data can come from an operational network or froma proposed/projected topology design. Likewise, the traffic demandscould come from measurements of the operational network, from estimatesor projections, or from customer subscriptions (e.g., for a virtualleased line service). The particular source of the network configurationand traffic demand information for the traffic engineering software isnot a limitation of the present invention.

[0016] 1. Data Model

[0017] The present invention advantageously combines diverse networkconfiguration information with diverse network measurements in a jointdata model. The following two subsections describe (a) a preferrednetwork topology model as well as practical ways of obtaininginformation to populate the topology model; and (b) a preferred trafficdemand model and various ways of obtaining network traffic measurementsto populate the traffic matrix.

[0018] A. Topology

[0019] Traffic engineering requires a network-wide view of theunderlying layer-three and layer-two topology. In accordance with apreferred embodiment of the present invention, a topology model ispresented that advantageously captures backbone connectivity,connections to customers and peers, link capacity, and OSPFconfiguration. A preferred data model includes data objects for networknodes and links in both a “pure” IP router layer (i.e. routers and layerthree links) and in a physical transport layer (i.e. devices andtrunks). It is advantageous to include layer two devices and trunks inthe data model because some networking technologies, such as FDDI andATM, introduce an intermediate switching fabric at layer two, e.g.multiple layer three links may share a single trunk, or a singlelayer-three link may correspond to a permanent virtual circuit (“PVC”)that traverses one or more ATM switches. This introduces layers ofconnectivity and capacity, which has implications for trafficengineering and reliability.

[0020]FIG. 2 sets forth a diagram of an IP backbone network 200 whichhighlights the different elements of the data model in the main routerlayer. IP routers and bi-directional layer-three links are representedas nodes and edges in FIG. 2. In a typical IP backbone network, eachrouter terminates a mixture of “access”, “peering”, and “backbone”links. An access link 225 connects directly to customers, e.g. to amodem bank for dial-up users, a web-hosting complex, or a particularbusiness or university campus. As shown in FIG. 2, some customers canhave two or more access links for higher capacity, load balancing orfault tolerance (such customers are referred to in the art as“multi-homed customers”). Peering links 215 connect the backbone networkto neighboring service providers, e.g. to a public Internet exchangepoint or directly to a private peer or transit provider. A typical ISPhas multiple peering links to each neighboring provider, typically indifferent geographic locations. Backbone links connect routers insidethe ISP backbone. The network in FIG. 2 has been simplified in that allaccess links terminate at Access Routers (“ARs” e.g. 221) and allpeering links terminate at Internet Gateway Routers (“IGRs” e.g. 211),and all remaining routers are characterized as Backbone Routers (“BRS”e.g. 201) that only terminate backbone links. In fact, in an operationalnetwork, this split in functionality simplifies the requirements foreach router: ARs provide high port density to connect to a large numberof customers with various access speeds and technologies; BRs providehigh packet-forwarding performance; IGRs can isolate peer traffic andsimplify management of inter-domain routing policies. (The meaning ofthe groupings X₁, X₂, . . . and Y₁, Y₂, . . . will be explained in thesection below on traffic demand).

[0021] Each router is represented by a data object with attributesincluding the router name, the loopback IP address of the router, thetype of the router (e.g. AR, BR, IGR), and the geographic location ofthe router in terms of city and latitude/longitude. In addition, eachrouter includes information about which links it originates. Forexample, a router data object can have the following parameters:routerName (string) Any uniquely identifying string. type (string) Thetype of router: e.g. BR = Backbone router; AR = Access router; IGR =Internet Gateway router; CR = Customer router; and PR = Peering router.count (int) useful router index, e.g. the third access router at Chicagowill have a count equal to 3. complex An identifier for a data objectrepresenting a central office housing the instant router. IP (string)The loopback interface of the router. dlatitude (double) Thedisplacement from the complex location for this router. dlongitude(double) The displacement from the complex location for this router.pnode An identifier for a data object representing the physical nodethat corresponds to this router. networks A sequence of IP masks. The IPnetworks that are directly connected to this router. Can be empty.

[0022] Each layer-three link is represented by a data object withattributes containing general information about the router originatingthe link, the name of the router card, the IP address of the interface,whether the link is shutdown or not, a textual description of itspurpose, its capacity, and its OSPF weight. Some attributes can beassociated with both directions of a link. For example, eachbidirectional link can be classified as an access, backbone, or peeringlink. Backbone links also belong to a particular OSPF area, which mustbe the same for both unidirectional links. Peering links are associatedwith a particular BGP peer, identified by its AS number and annotated bythe IP address of the BGP peer in the remote domain.

[0023] For example, a link data object can model a router interface andhave the following parameters. Note that only interfaces that terminateat another router are included, e.g. interfaces that correspond to a PVCbut not the interfaces that terminate on an ATM switch: x An identifiercorresponding to the router data object with this interface. CardName(string) Name of the interface. y An identifier corresponding to therouter data object where this link terminates. linkid (int) This givesan identifier on the full-duplex link. This can be used to find the linkin the opposite direction. type (string) The type of the link, e.g.BACKBONE, INTERNAL, or ACCESS. IP (string) The IP address of theinterface. Capacity (int) The bit rate of the interface in Kbytes. Notethat for some links, e.g. PVCs, this number makes not too much sensesince the bandwidth is shared between multiple interfaces. OSPF (int)The OSPF weight of the interface. OSPFarea The OSPF area of theinterface. Status (int) Status of the interface. Desc (string) Thedescription field from the router configuration file. plinks A sequenceof references to physical link data objects. This is a list of thephysical links that this link lays out to.

[0024] Each device or physical node is represented by a data object thathas parameters identifying what type of device it is, e.g. a router oran ATM switch, where the node is located, and a list of trunks thatoriginate at the device. Trunks describe the connectivity betweenrouters and devices, and include the information about which linkstraverse a given trunk. For example, each physical node (“pnode”) canhave the following parameters: name (string) An identifying string forthe physical node. complex An identifier for a data object representinga central office housing the physical node. dlatitude (double) Thedisplacement from the complex location for this pnode. dlongitude(double) The displacement from the complex location for this pnode. type(string) The type of pnode, e.g. router or ATM.

[0025] And each physical link (“plink”) can have the followingparameters: x An identifier corresponding to the pnode data object wherethis link originates. CardName (string) Any string which with x uniquelyidentifies the plink. y An identifier corresponding to the pnode dataobject where this link terminates. trunkid (int) This gives anidentifier on the trunk. type (string) capacity (int) Desc (string)

[0026] The above model is very general and its objects can be populatedin a number of different ways, such as modifying an existing data model,constructing an artificial network, or extracting the information fromthe real network.

[0027] Extracting Network Topology. Unfortunately, there is no singleplace within a typical IP network that would allow extraction of theinformation necessary to populate the above preferred model. Rather, theinformation is distributed among many routers in the Internet. Evenwithin an ISP network, information is decentralized. For example, evenOSPF link state information may be insufficient to extract the topologyof an IP backbone, especially since OSPF areas hide information. Inaddition, OSPF link state certainly does not contain information aboutaccess links and peering links.

[0028] End-to-end mechanisms such as “ping” and “traceroute” can be usedfor basic network topology discovery—but are cumbersome and provide onlybasic connectivity information. SNMP queries or traps can also beutilized, but require active querying of all network elements. See,e.g., “A Simple Network Management Protocol (SNMP),” IETF RFC 1157,Network Working Group, May 1990.

[0029] An alternative approach is to extract the information from therouter configuration files for the operational network. This has theadvantage of capitalizing on all the additional information contained inthe router configuration files, including customer and peer information.A perhaps less obvious advantage is that the router configuration filesare routinely logged for backup purposes and easily accessible withoutaccessing the live operational network. The disadvantages are that theinformation is not updated continuously and that the configuration filesreflect the state of the network absent failures. Nevertheless, routeror link failures or physically disconnected links can be taken accountof by a separate data feed. Populating the data objects in the datamodel using the router configuration files is made much easier by usinga packet-switched network configuration debugger and database, asdescribed in co-pending utility patent application, “Netdb: IP NetworkConfiguration Debugger/Database,” U.S. Patent and Trademark Officeapplication Ser. No. 60/160,446.

[0030] Traffic engineering requires information about the IP addressesreachable from each access and peering link. Reachability informationcan be obtained from a number of different sources in the network, e.g.,as described below, forwarding tables, BGP tables in general, and routereflector BGP tables in particular. The inventors have found itadvantageous to rely on the forwarding tables, although the sameinformation could come from other sources as well, such as the BGPtables, configuration files, etc.

[0031] The packet-forwarding tables at each of the Access Routers may beused to extract customer IP addresses, when not listed in a routerconfiguration file. The forwarding table is, in a way, the ultimateauthority for how the backbone forwards packets to a set of customer IPaddresses. The forwarding table can be logged periodically (e.g., withthe IOS command “show ip cef”) and post-processed (e.g. using a Perlscript) to extract the set of network addresses associated with eachaccess link. The table includes three main fields—the network address,the next-hop IP address (when known), and the card name of the outgoinglink. The network address can be associated with the appropriate accesslink based on the card name, which is part of the topology model that isextracted from the router configuration files.

[0032] The BGP routing table may also be processed to determine whichpeering links are used to reach each external IP address. An ISP haslimited control over the external IP addresses that connect to theInternet through other service providers. Routing of traffic from theseexternal addresses depends on the policies other service providersemploy for selecting paths and propagating router advertisements.Routing of traffic from customers to these external addresses depends onthe advertisements the ISP receives and how they are processed. Applyinglocal policy to the route advertisements results in a BGP routing tablethat indicates the chosen AS path for each external network address.See, e.g., “A Border Gateway Protocol (BGP-4),” IETF RFC 1771, NetworkWorking Group, March 1995. Based on this information, the set of peeringlinks that can be used to reach each external network address can bedetermined. Similar to the customer addresses associated with eachaccess link, each peering link can be associated with a set of externalnetwork addresses (it should be noted that, in a preferred embodiment ofthe present invention, this information is used to study routing oftraffic destined for that network address and not how traffic form thatnetwork address enters the network).

[0033] The BGP routing table from a single route reflector in thebackbone can also be utilized to determine the set of peering linksassociated with each external network address (e.g., using the IOScommand “show ip bgp”). The ARs and BRs receive advertisements of the ASpaths selected by the IGRs. Given the potentially significantfluctuations in BGP routing information, it is advantageous toincorporate a continuous feed of BGP information into the model. Eachentry in the BGP routing table corresponds to a single IGR that can beused to reach a particular network address. An entry in the tableindicates the network address, the loopback address of the associatedIGR, and the AS path. A simple Perl script may be used to process all ofthe entries in the BGP table to determine the set of network addressesassociated with each peering link.

[0034] B. Traffic Demand

[0035] Effective traffic engineering requires not just a view of thetopology but also an accurate estimate of the offered load betweenvarious points in the backbone. How should traffic demands be modeledand inferred from operational measurements? At one extreme, IP trafficcould be represented at the level of individual source-destinationpairs, possibly aggregating sources and destinations to the networkaddress or AS level. Representing all hosts or network addresses,however, would result in an overly large traffic matrix, virtuallyimpossible to populate since no single ISP is likely to see all of thetraffic to and from each network address. Alternatively, IP trafficdemands might be aggregated to point-to-point demands between edge linksor routers in the ISP backbone. This approach, however, has fundamentaldifficulties in dealing with interdomain traffic (traffic whose ultimatedestination belongs to another domain). Inter-domain traffic, whichconstitutes a large fraction of traffic in operational IP networkstoday, may exit the ISP backbone from any of a set of egress links,determined by interdomain routing policies. Modeling interdomain trafficas point-to-point would couple the demand model to internal routingconfiguration, making it highly problematic to predict how changinginternal routing configuration would influence network load.

[0036] In accordance with an aspect of the present invention, analternative model is described which effectively handles interdomaintraffic and advantageously is invariant to changes in the internalrouting configuration. The preferred model of traffic demand consists ofan ingress link, a set of egress links, and a volume of load. Forexample, the traffic demands between routers can be represented as dataobjects with the following attributes: x An identifier corresponding tothe source router data object. y A sequence of router data objectsrepresenting the set of potential destination routers. The string x, ymust uniquely identify this demand. Kbytes (double) The amount of datafor this demand. packets (double) Alternative measurement of demand.Does not need to be used. arrivals (double) Alternative measurement ofdemand. Does not need to be used.

[0037] The path traveled by an IP packet depends on the interplaybetween interdomain routing protocols (e.g. BGP) and intradomain routingprotocols (e.g. OSPF, IS-IS, or MPLS). The ISP network lies in themiddle of the Internet and may not have any direct connection to thesender or the receiver of any particular flow of packets. As such, aparticular destination prefix may be reachable via multiple egress linksfrom the ISP: e.g. a multi-homed customer may receive traffic onmultiple links that connect to different points in the backbone or anISP may have multiple links connecting to a neighboring provider. Theultimate decision of which route to use depends on the BGProute-selection process. By associating each traffic demand with a setof egress links that could carry the traffic, the set basicallyrepresents the outcome of the early stages of the BGP route selectionprocess before the consideration of the intradomain protocol.

[0038] The set of peer links can be represented by a logical node X_(i),and, similarly, a set of access links can be represented by a logicalnode Y_(j), as illustrated in FIG. 2. The matching process can draw onthe list of customer network addresses from the Access Router forwardingtables and the external network addresses from the BGP table. The sourceand destination addresses can be aggregated by performing alongest-prefix match on these lists of network addresses. The networkaddresses can then be used to associate traffic measurements with theappropriate sets of access or peering links.

[0039] Traffic Measurement. It is advantageous to collect trafficmeasurements at all ingress links to compute traffic demands andidentify the traffic as it enters the ISP backbone. Collectingpacket-level traces at each ingress link, however, would beprohibitively expensive. Instead, flow-level statistics can be collectedby each ingress router, a “flow” being defined in the art as a set ofpackets that match in the key IP and TCP/UDP header fields (such as thesource and destination address, and port numbers) and arrive on the sameingress link. For example, routers manufactured by Cisco have a Netflow™feature that, when enabled, permits the router to keep track of theamount of traffic in each active flow. The router can summarize thetraffic statistics on a regular basis, either after the flow has becomeinactive or after an extended period of activity. Sampling the flowmeasurements may also be performed to reduce the total amount of data.

[0040]FIG. 3 sets forth an algorithm, in pseudo-code, for computing thetraffic demands upon receiving a flow record with the followinginformation: an input link and a destination IP address dest to identifythe end-points of the demand, the start and finish times of the flow,and the total number of bytes in the flow. Additional information in themeasurement records, such as TCP/UDP port numbers or type-of-servicebits, could be used to compute separate traffic demands for eachquality-of-service class or application. Aggregating the flow-levelmeasurements into traffic demands requires information about thedestination prefixes associated with each egress link. For example, theaggregation process draws on a list, dest_prefix_set, of networkaddresses, each consisting of an IP address and mask length. Eachdestination prefix, dest_prefix, can be associated with a set of egresslinks, reachability (dest_prefix). For example, in an operationalnetwork, these prefixes could be determined from the entries in theforwarding tables at routers that terminate egress links. (Note that theforwarding table at a router connected to an ingress link could have adifferent set of prefixes, particularly if the IP routing protocols havebeen configured to aggregate subnet address). In particular, eachforwarding table entry identifies the next-hop link(s) for a particularprefix. This enables identification of the prefixes associated with eachegress link.

[0041] As reflected in FIG. 3, computing traffic demands across acollection of flows at different routers introduces a number of timingchallenges. The flow records do not capture the timing of the individualpackets within a flow. Nevertheless, traffic engineering occurs on atime scale larger than most flow durations, thus permitting time to bedivided into consecutive width-second bins in which most flows willstart and finish. When a flow spans multiple bins, the traffic can besubdivided in proportion to the fraction of time spent in each timeperiod

[0042] An alternative to measuring traffic demand at each ingress linkis to collect measurements at a much smaller number of edge links, e.g.the links connecting the ISP to neighboring providers. This isadvantageous in that it frees access routers, which often may not becapable of collecting fine-grain measurements, from the additionalmeasurement overhead. In contrast, the small number of high-end routersthat connect neighboring providers typically have a much smaller numberof links, with substantial functionality (including measurementfunctions) implemented directly on the interface cards that connect eachlink to the router. By monitoring both the ingress and egress links atthese locations, it is possible to capture a large fraction of thetraffic in the ISP backbone—but this introduces new complications formeasuring traffic.

[0043] Traffic flows in the IP backbone can be characterized as“inbound” traffic (i.e. packets travelling from a peering link to anaccess links), “transit” traffic (travelling between two peering links),“outbound” traffic (travelling from an access link to a peering link)and “internal” traffic (travelling between two access links). Thecharacterization of the traffic flow will affect how the flow should behandled. As further described below, the flow can be classified at apeering link based on the input and output links as follows: InputOutput Classification Action Peer Backbone Inbound or multi-hopPoint-to-multipoint transit demand Peer Peer Single-hop transitPoint-to-multipoint demand Backbone Backbone Backbone traffic Omit flowBackbone Peer Outbound or multi-hop Identify possible ingress transitlink(s). Omit flow or compute demand.

[0044] a. Internal Traffic. It should be noted that monitoring thepeering links does not capture internal traffic sent from one accesslink to another. For customer traffic to and from particularly importantaccess links (e.g., to the ISP's e-mail, Web, and DNS services), thiscan be addressed by enabling flow-level measurements—effectivelytreating these connections like peering links.

[0045] b. Inbound Traffic. For inbound flows, traveling from a peeringlink to a backbone link, the above measurement methodology can bedirectly applied, since flow-level measurements are available from theingress link. FIG. 4 sets forth the process of aggregating the flowrecords, skipping the details from FIG. 3 regarding dividing the bytesof the flow across multiple time_bins.

[0046] c. Transit Traffic. Transit traffic falls into twocategories—single-hop and multiple-hop. A single-hop flow enters andexits the ISP backbone at the same edge router, without traversing anybackbone links: in this case, the flow can be measured once, at thisrouter. A multi-hop flow enters at one router, traverses one or morebackbone links, and exits at another router. Measuring both ingress andegress traffic at the peering links, thus, results in duplicatemeasurements of transit traffic that travels from one provider toanother; special attention is required to avoid double-counting thistraffic. The best place to capture a transit flow is at its ingresslink, where the above methodology can be applied. To avoiddouble-counting the flow, the flow records generated by multi-hoptransit flows as they leave the network need to be discarded. Thisrequires distinguishing outbound flows (introduced by an access link)from transit flows (introduced by a peering link). For a flow leavingthe ISP network, the algorithm in FIG. 4 attempts to match the source IPaddress with a customer prefix at an access link. For transit flows,this matching process would fail, and the associated flow record wouldbe properly excluded.

[0047] d. Outbound Traffic. Computing the outbound traffic demands thattravel from across links to peering links becomes more difficult, sinceflow-level measurements are not available at the ingress links. The flowmeasurements provide two pieces of information that help to infer theaccess link responsible for the outbound traffic (1) the source IPaddress and (2) the input/output links that observed the flow at theegress router. The source IP address indicates which customer generatedthe traffic (assuming the sender has not spoofed the source address).The source IP address should be matched with a customer prefix which, inturn, should be matched with a set of possible access links that couldhave generated the traffic. The pseudocode in FIG. 4 draws on a list,src_access_prefix_set, of the network addresses introducing traffic ataccess links. Each source prefix, src_pref ix, can be associated with aset of ingress links, sendability (src_prefix). It should be noted thatthe routing forwarding tables are not sufficient for identifying thesource addresses that might generate traffic on an access link. This isbecause Internet routing is not symmetric: traffic to and from acustomer does not necessarily leave or enter the backbone on the samelink. Fortunately, an ISP typically knows the IP addresses of itsdirectly-connected customers, and, in fact, may assign IP prefixes froma larger address block belonging to the ISP. Packet filters are oftenused by ISPs to remove traffic with bogus source IP addresses, and,these packet filters are specified in the router's configuration filewhich may be accessed and parsed to determine which source prefixes toassociate with each access link. From this information it can bedetermined the set of access links associated with each source prefix.(Where customers connect to other service providers or have downstreamcustomers of their own, it may be preferable to perform flow-levelmeasurements at the ingress links rather Man depending on knowing theset of links where these sources could enter the ISP backbone).

[0048] Information about the input and output links that measured theflow should be maintained, as this information is useful to help inferwhich of the access links could have generated the traffic. Thealgorithm in FIG. 4 results in a point-to-multipoint demand for inboundand transit flows. Each outbound flow, however, is associated with a setof ingress links, resulting in a multipoint-to-multipoint aggregate.Computing point-to-multpoint demands for outbound traffic requires anadditional step to determine which access link initiated the traffic.FIG. 5 sets forth a “disambiguation” process which attempts to determinewhether an outbound flow could have entered the network at a giveningress link based on knowledge of the backbone topology and intradomainrouting configuration at the time the flow was measured. Information onthe possible paths from each ingress link to each egress link isobtained from a routing model that is further described below in Section2. For a given ingress link and set of egress links, it is determined onwhich egress link the flow would exit the network. If this was not theegress link where the flow was observed, then this ingress link can beeliminated from consideration. Knowing the path(s) from the ingress linkto the egress link provides additional information: where the path ofthe flow from the ingress link does not include both of the links thatobserved the flow (i.e. the input backbone link and the output peeringlink), the ingress link should again be excluded from consideration. Theprocess should be repeated for each of the possible ingress links, asshown in FIG. 5. The process has three possible outcomes. First, asingle ingress link could have generated the traffic, resulting in theideal situation of a single point-to-multipoint demand. Second, morethan one of the candidate ingress links could have generated thetraffic, in which case the disambiguation process generates multipledemands, each with an equal fraction of the traffic. Third, if none ofthe candidate ingress links could have generated the traffic, thedisambiguation process has failed and the flow record is discarded. Thisprovides a useful consistency check on the initial processing of theflow-level data.

[0049] 2. Routing Model

[0050] A feature of the preferred embodiment of the present invention isthat it combines the network model and the traffic measurements with anaccurate model of path selection. Specifically, a routing moduledetermines the path(s) chosen by the relevant routing protocol for eachtraffic demand, and the load imparted on each link as the traffic flowsthrough the network. The routing module captures the selection ofshortest paths to/from multi-homed customers and peers, the splitting oftraffic across multiple shortest-path routes, and the multiplexing oflayer-three links over layer-two trunks. These capabilities allow a userto explore the impact of changes in the traffic demands or in theunderlying network topology.

[0051] There are a variety of routing protocols that may be utilizedwith the present invention, e.g., OSPF, IS-IS, etc. For example, theOSPF protocol defines how routers within an area exchange link-stateinformation and compute shortest paths based on the sum of the linkweights. See “OSPF Version 2”, IETF RFC 2328, Network Working Group,April 1998. The link weights are static and are typically configuredbased on the link capacity, physical distance, and some notion of theexpected traffic load. The chosen paths do not change unless a link orrouter failure occurs, or the OSPF parameters are reconfigured. Theseare rare events, particularly for the backbone links that participate inthe routing protocol. As such, the routing module can consider a singleinstance of the network topology and OSPF configuration and need notsimulate the details of the OSPF protocol, such as the flooding oflink-state advertisements or the exchange of “hello” messages. Therouting module can be verified by comparing the resulting paths with therouter forwarding tables or traceroute experiments on an operationalnetwork Performing the path selection computation inside the tool,rather than using the forwarding tables or traceroute results directly,facilitates experimentation with alternate OSPF configurations anddifferent topologies.

[0052] When all of the backbone links reside in a single OSPF area, pathselection simply involves computing the shortest paths between each pairof routers, based on the link weights. In a hierarchical network,traffic between two routers in the same area follows a shortest pathwithin the area, even if the network has a shorter path that involveslinks in other areas. When traffic must travel between routers indifferent areas, the path depends on how much information each area hasabout its neighbors. The routing module can assume that the network doesnot summarize routing information at area boundaries. In the absence ofroute summarization, each border router reports the cost of the shortestpath(s) to each of the other routers in the area, and the trafficbetween routers in different areas simply follows a shortest pathwithout regard to the area boundaries. The routes can be computed using,for example, Dijkstra's shortest-path-first algorithm, which iswell-known in the art. To limit the computational overheads, animplementation of the routing module can operate on a reduced networkgraph that collapses equivalent edges and nodes, and avoids recomputingdistances and paths by caching intermediate results.

[0053] Path selection becomes more complex when there are multipleshortest paths between a pair of routers. Such ties arise very naturallywhen the network topology has parallel links between adjacent routersfor additional capacity. Ties also surface when many of the links in thenetwork have similar weights. This is sometimes done intentionally toincrease the effective capacity between two endpoints. The presence ofmultiple shortest paths allows for load-balancing of the traffic betweenthe two endpoints. This is achieved by allowing the IP forwarding tableto have multiple outgoing links associated with a single entry. Ratherthan alternating between these links at the packet level, routerstypically attempt to forward packets for the same source-destinationpair along a single path; this reduces the likelihood that packets fromthe same TCP connection arrive out-of-order at the receiver.Load-balancing is typically achieved by performing a hash function onthe source and destination IP addresses of each packet. The value of thehash function determines which outgoing link should carry the packet.

[0054] The details of the “tie-breaking” function can be modeled in therouting module. This, however, significantly complicates the pathselection computation and would require computing traffic demands at asignificantly finer level of granularity. In addition, the details ofthe hashing function, and how the outputs of the has function map toparticular outgoing links are not specified by the OSPF protocol and, assuch, depend on the vender's implementation. Fortunately, these detailsare not important. The hash function is designed to support an evensplitting of the traffic across the multiple outgoing links, especiallyfor backbone links that carry a diverse mixture of traffic withdifferent source and destination addresses. As such, the routing moduleadvantageously splits traffic evenly cross each of the outgoing linksalong a shortest path. For example, with regard to FIG. 6, if a routerhas two outgoing links on shortest paths, each link would carry 50% ofthe traffic. The division of traffic is recursive, with the downstreamrouters dividing the traffic across each of their outgoing links, as setforth in FIG. 6. For a more conservative estimate of the load on eachlink, the routing module could assume that each outgoing link carries alittle extra more than its fair share of the traffic by applying amultiplicative factor.

[0055] Using the traffic demands described in the previous section, therouting module can operate on a set of demands, each traveling from onepeering or access link to a set of access or peering links. The modulecomputes the set of shortest-path routes based on the topology and theOSPF configuration, and determines how the demand splits across themultiple paths. Repeating this process for each demand results in anestimate of the load imparted on each link. Then, the routing moduledetermines the load on each trunk (layer-two link) by summing across theassociated layer three links. The generality of the routing modelfacilitates experiments with alternate topologies and OSPFconfigurations, as illustrated in the next section. It also supportsexperimentation with the BGP policies for outbound traffic, by changingthe sets of peering links associated with external network addresses.

[0056] 3. Visualization

[0057] A graphical user interface, such as the one set forth in FIGS. 7through 11, can be used to provide an efficient visualizationenvironment with many ways to explore the data in the data model. As setforth above, each router and link is modeled with a data object. FIG. 7sets forth an example of an information panel which displays attributesfor objects of a given type. The information panel in FIG. 7 permits auser to quickly scroll through a list of links and see the correspondingattributes to the selected link in the bottom part of the panel, alongwith corresponding physical links in the right hand box in the panel.

[0058] It is useful to permit the data model to associate statisticalinformation with objects. Each statistic need be no more than simply avalue for each object of some type. For example, a link utilizationstatistic, which is a percentage associated with each link, can becalculated and displayed as set forth in FIG. 8. There should be norestriction on how many statistics can be associated with an objecttype. Thus, link utilization statistics can be stored for periods oftime and can be used to create histograms, scatter plots, tables, etc.The color or size of the object when displayed can be utilized toreflect the statistic, thereby providing a visual representation of thestatistic, e.g. coloring links with high utilization as thick and red.

[0059] It is advantageous to include some search facility permittingqueries on objects, as set forth in FIG. 9. The “Find Link” userinterface in FIG. 9 permits arbitrary expression searches involvingstatistics, object fields, special filters, etc.

[0060]FIG. 10 and 11 sets forth the basic display of the router and linkdata objects graphically superimposed over a map of the relevantgeography. Given the large numbers of nodes and links that may need tobe displayed, it is helpful to permit the user to choose which sets ofobjects to display as well as to define different layers of aggregationand abstraction, e.g. combining routers into complexes, aggregatingparallel links, etc. The display in FIG. 10 and 11 has a user interfacethat recognizes the notion of a current object for each type. An objectbecomes the current object either when it is selected or when the mouseis moved on top of it graphically. For example, a user whose mousepointer hovers over a particular link displayed in FIG. 10 would causethe associated Link Panel window and Link Statistics window to change toinformation regarding that particular link.

[0061] It is advantageous for the visualization module to permit changesto the data model “on the fly” such as modifications to an OSPF weightof a link in the network. Then the software can use the routing moduleto automatically recalculate all routes for all active traffic demands,and update all relevant statistics that are based upon the trafficincluding link load and utilization. It is also helpful to maintain atleast two different sets of weights, one that can be manipulated and onethat can act as an anchor or baseline.

[0062] The foregoing Detailed Description is to be understood as beingin every respect illustrative and exemplary, but not restrictive, andthe scope of the invention disclosed herein is not to be determined fromthe Detailed Description, but rather from the claims as interpretedaccording to the full breadth permitted by the patent laws. It is to beunderstood that the embodiments shown and described herein are onlyillustrative of the principles of the present invention and that variousmodifications may be implemented by those skilled in the art withoutdeparting from the scope and spirit of the invention. For example, thedetailed description described the present invention in the context ofthe Internet and IP-based backbone networks. However, the principles ofthe present invention could be extended to other types ofpacket-switched networks. Such an extension could be readily implementedby one of ordinary skill in the art given the above disclosure.

What is claimed is:
 1. A computer readable medium containing executableprogram instructions for performing a method on a computer connected toa network comprising the steps of: receiving network topologyinformation as an input; receiving network traffic demand information asan input; constructing a data model of a packet-switched network fromthe network topology information and network traffic demand informationwherein the data model further comprises data objects for network nodes,network links, and for network traffic demands; and constructing arouting model wherein the data objects for network nodes, network links,and for network traffic demands are utilized to simulate network trafficin the packet-switched network.
 2. The computer readable medium of claim1 wherein the network topology information is derived from data obtainedfrom an operational packet-switched network.
 3. The computer readablemedium of claim 2 wherein the data is extracted from routerconfiguration files.
 4. The computer readable medium of claim 2 whereinthe data is extracted utilizing end-to-end query mechanisms.
 5. Thecomputer readable medium of claim 1 wherein the network topologyinformation is derived from a proposed topology design.
 6. The computerreadable medium of claim 1 wherein the network traffic demandinformation is derived from data obtained from an operationalpacket-switched network.
 7. The computer readable medium of claim 6wherein the data is extracted from traffic measurements collected atingress routers.
 8. The computer readable medium of claim 7 wherein thetraffic measurements are made between an ingress link and a set ofegress links.
 9. The computer readable medium of claim 8 wherein thetraffic measurements are collected by associating one or moredestination network addresses with the set of egress links.
 10. Thecomputer readable medium of claim 9 wherein the set of egress links isidentified by extracting reachability information from networkforwarding tables.
 11. The computer readable medium of claim 9 whereinthe set of egress links is identified by extracting reachabilityinformation from BGP tables.
 12. The computer readable medium of claim 9wherein the set of egress links is identified by extracting reachabilityinformation from network configuration files.
 13. The computer readablemedium of claim 1 wherein the network traffic demand information isderived from estimates of projected network traffic demand.
 14. Thecomputer readable medium of claim 1 wherein the network traffic demandinformation is derived from customer subscription information.
 15. Thecomputer readable medium of claim 1 further comprising the step ofproviding an interface to the data model that graphically displays thenetwork nodes, network links and network traffic calculated by therouting model.
 16. The computer readable medium of claim 1 wherein therouting model simulates the OSPF routing protocol.
 17. The computerreadable medium of claim 1 wherein the routing model simulates the IS-ISrouting protocol.
 18. A method of traffic engineering in apacket-switched network comprising the steps of: retrieving networktopology information; retrieving traffic measurement information;constructing a data model of a packet-switched network from the networktopology information and network traffic information wherein the datamodel further comprises data objects for network nodes, network links,and for network traffic demands; and constructing a routing modelwherein the data objects for network nodes, network links, and fornetwork traffic demands are utilized to simulate network traffic in thepacket-switched network.
 19. The method of claim 18 wherein the networktopology information is derived from data obtained from an operationalpacket-switched network.
 20. The method of claim 19 wherein the data isextracted from router configuration files.
 21. The method of claim 19wherein the data is extracted utilizing end-to-end query mechanisms. 22.The method of claim 18 wherein the network topology information isderived from a proposed topology design.
 23. The method of claim 18wherein the network traffic demand information is derived from dataobtained from an operational packet-switched network.
 24. The method ofclaim 23 wherein the data is extracted from traffic measurementscollected at ingress routers.
 25. The method of claim 24 wherein thetraffic measurements are made between an ingress link and a set ofegress links.
 26. The method of claim 25 wherein the trafficmeasurements are collected by associating one or more destinationnetwork addresses with the set of egress links.
 27. The method of claim26 wherein the set of egress links is identified by extractingreachability information from network forwarding tables.
 28. The methodof claim 26 wherein the set of egress links is identified by extractingreachability information from BGP tables.
 29. The method of claim 26wherein the set of egress links is identified by extracting reachabilityinformation from network configuration files.
 30. The method of claim 18wherein the network traffic demand information is derived from estimatesof projected network traffic demand.
 31. The method of claim 18 whereinthe network traffic demand information is derived from customersubscription information.
 32. The method of claim 18 further comprisingthe step of providing an interface to the data model that graphicallydisplays the network nodes, network links and network traffic calculatedby the routing model.
 33. The method of claim 18 wherein the routingmodel simulates the OSPF routing protocol.
 34. The method of claim 18wherein the routing model simulates the IS-IS routing protocol.